ERNIE 2.0: A Continual Pre-Training Framework for Language Understanding
نویسندگان
چکیده
منابع مشابه
A New Pre-Training Method for Training Deep Learning Models with Application to Spoken Language Understanding
We propose a simple and efficient approach for pre-training deep learning models with application to slot filling tasks in spoken language understanding. The proposed approach leverages unlabeled data to train the models and is generic enough to work with any deep learning model. In this study, we consider the CNN2CRF architecture that contains Convolutional Neural Network (CNN) with Conditiona...
متن کاملA multimedia framework for effective language training
We present a novel framework for the multimodal display of words using topological, appearance, and auditory representations. The methods are designed for effective language training and serve as a learning aid for individuals with dyslexia. Our topological code decomposes the word into its syllables and displays it graphically as a tree structure. The appearance code assigns color attributes a...
متن کاملToward a Formal Framework for Continual Learning
This paper revisits the continual-learning paradigm I described at the previous workshop in 1995. It presents a framework that formally merges ideas from reinforcement learning and inductive transfer, potentially broadening the scope of each. Most research in RL assumes a stationary (non-changing) world, while research in transfer primarily focuses on supervised learning. Combining the two appr...
متن کاملAdaptive Training for Robust Spoken Language Understanding
Spoken Language Understanding, as other areas of Language Technologies, suffers from a mismatching between the conditions of the training of the models and the real use of the systems. If the semantic models are estimated from the correct transcriptions of the training corpus, when the system interacts with real users, some recognition errors can not be recovered by the understanding system. To...
متن کاملAbductive Reasoning for Continual Dialogue Understanding
In this paper I present a continual context-sensitive abductive framework for understanding situated spoken natural dialogue. The framework builds up and refines a set of partial defeasible explanations of the spoken input, trying to infer the speaker’s intention. These partial explanations are conditioned on the eventual verification of the knowledge gaps they contain. This verification is don...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Proceedings of the AAAI Conference on Artificial Intelligence
سال: 2020
ISSN: 2374-3468,2159-5399
DOI: 10.1609/aaai.v34i05.6428